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Abstract 


The  generalized  programming  algorithm  consists  of  major  iterations 
and  minor  iterations;  each  major  iteration  consists  of  manv  minor 
iterations.  A  major  iteration  terminates  with  the  addition  of  a  Gomory 
cut.  A  minor  iteration  corresponds  to  the  solution  of  a  master  linear 
programming  problem  and  a  group  optimization  problem.  Activities  can 
be  deleted  at  the  end  of  each  minor  iteration  as  the  result  of  new 
lower  bounds  generated  during  the  minor  iteration.  The  algorithm 
allows  most  of  the  integer  programming  columns  to  be  kept  in  secondary 
storage  and  evaluated  in  core  one  at  a  time. 


I 


srsosa 


Introduction 

One  of  the  major  computational  problems  of  linear  programming  (LP) 
and  integer  programming  (IP)  is  dealing  with  the  large  number  of  variables 
(also  referred  to  as  activities  or  columns)  arising  in  many  applications. 
For  LP,  the  difficulty  is  due  mainly  to  the  limitations  of  core  storage. 
For  branch  and  bound  IP  algorithms  ([5],  [10],  [15],  [17]),  however,  there 
is  the  additional  difficulty  of  greatly  increasing  computer  solution  time 
as  a  function  of  the  number  of  variables  in  the  branch  and  bound  search. 

The  problem  of  many  columns  in  LP  has  been  solved,  at  least  in  part, 
by  the  generalized  programming  method  of  Dantzig  and  Wolfe  ([2]).  This 
method  can  be  viewed  as  a  generalized  Lagrange  multiplier  (GLM)  technique 
where  the  optimal  simplex  multipliers  from  a  series  of  LP  approximations 
are  used  to  indicate  attractive  (low  reduced  cost)  activities.  Our  puroose 
here  is  to  provide  a  similar  generalized  programming  algorithm  for  IP 
which  makes  it  possible  for  most  of  the  columns  of  a  given  IP  problem  to 
be  kept  in  secondary  storage  and  evaluated  in  core  one  at  a  time.  Carried 
to  the  extreme,  our  orocedure  will  reduce  the  original  set  of  activities  to 
a  set  of  optimal  activities,  and  indicate  their  optimal  non-negative  integer 
levels.  It  would  appear,  however,  that  the  generalized  nrogramming  algorithm 
would  best  be  combined  with  some  branch  and  bound  algorithm. 

The  problem  we  wish  to  study  is:  Find  z*  where 

n 
z*  =  min   E  c .X . 
j=l  ^  ' 
n 
s.t.   E  a.  .X.  =  b. ,  i=l ,...,m  (1) 

j=l  '^  ^         ' 

X.  non-negative  integer 


where  the  coefficients  c-,  a..,  b.  are  integer,  and  n  is  a  large  (finite) 

number.  An  activity  a.  is  an  m  vector  with  components  a...  At  any  point 

during  the  computation,  we  let  x  denote  the  incumbent;  that  is,  x  is  the  least 

n 
cost  solution  found  thus  far.  The  incumbent  cost  i     ex.  is  denoted  by 

j=l  ^  ^ 
z.  If  an  incumbent  is  not  known,  it  is  usual  in  practice  to  guess  at  a 

safe  upper  bound  value  z  on  problem  (1),  we  will  assume  this  to  be  the 

case. 

There  are  in  fact  tv;o  classes  of  problems  to  be  studied.  The  first 
is  the  one  in  which  the  activities  a.  in  (1)  have  all  been  generated  nrior 
to  optimization.  The  other  class  consists  of  those  oroblems  where  columns 
are  generated  iteratively  during  optimization  from  subprograms,  usually 
dynamic  progranming  subproblems  (e.g.,  cutting  stock  [7],  traveling 
salesman  [13],  multi-item  inventory  [3],  and  multi -commodity  flow  problems 
[4]).  The  latter  class  will  be  discussed  in  a  later  paper. 

Let  LP(1)  denote  problem  (1)  with  the  integrality  restriction  relaxed; 
for  clarity,  let  IP(1 )  denote  oroblem  (1).  We  try  to  solve  LP(1)  by 
finding  an  optimal  LP  basis,  either  in  the  conventional  way  or  by 
generalized  programming.  If  the  computational  effort  required  to  find 
an  optimal  LP  basis  is  too  great,  any  LP  basis  will  suffice.  This  feature 
is  important  when  generalized  programming  is  used  to  solve  LP(1)  because 
of  its  slow  convergence  characteristics  ([7],  [13;  p.  1146]).  For 
expositional  convience,  however,  suppose  the  optimal  basis  B  is  found 


and  that  B  =  (a-,,  a^j-.-.a  ).  Suppose  further  that  the  non-negative  vector 
b  =  B  b  has  some  non-integer  components.  Let  tt  =  CpB   where  Cn=(c,,...,c  ) 
and  N  =  (a  , ,. . .  ,a  ).  The  basis  B  is  optimal  because  c .  -  Tia .  >^  0,  j=m+l ,. . .  ,n, 
For  notational  ourposes  let  "zq  =  Tib,  a".  =  B'  a-,  T.  =  c.   -   -na,  j=m+l,..,,n. 


The  generalized  Drogramming  algorithm  to  be  given  below  consists  of 
major  iterations  and  minor  iterations;  each  major  iteration  consisting  of 
many  minor  iterations.  A  major  iteration  begins  with  an  optimal  (or 
feasible)  solution  to  LP(1).  Additional  conditions  are  generated  which 
must  be  satisfied  by  the  integer  solutions  to  (1).  These  solutions  are 
then  used  to  generate  a  particular  Gomory  cut  and  the  cut  is  added  to  the 
optimal  tableau  for  LP(1).  The  major  iteration  ends  when  the  LP  has 
been  re-optimized.  We  describe  below  one  major  iteration  of  the  generalized 
programming  algorithm;  all  subsequent  iterations  are  identical. 

The  particular  cut  we  seek  during  a  major  iteration  is  very  closely 
related  to  the  one  generated  by  the  method  given  in  [16].  The  cut  in  [16] 
is  shown  to  have  the  following  strong  property:  The  addition  of  the  cut 
to  a  given  LP  produces  as  great  an  increase  in  the  objective  function  as 
can  be  achieved  by  conditions  deduced  from  the  fractional ity  of  the  LP 
basis  and  solution.  The  generalized  programming  method  of  [16]  consists 
of  the  iterative  solution  of  a  master  LP  and  a  shortest  route  group  optimization 
problem.  A  minor  iteration  is  one  in  which  the  master  passes  prices  to  the 
shortest  route  problem,  and  the  shortest  route  problem  passes  a  solution  to 
the  master. 

In  the  next  section  we  review  the  reformulation  of  IP(1)  used  in  [16] 
to  generate  strong  cuts  and  show  how  this  method  can  be  made  to  handle 
efficiently  many  columns.  In  addition,  we  discuss  briefly  the  many  possible 
problem  manipulations        of  the  reformulation.  The  strateay  of  the 
generalized  programming  algorithm  can  be  classified  as  dualization  outer 
linearization/relaxation  by  the  taxonomy  of  Geoffrion  [6;  p.  656],  and 
the  manipulations  we  discuss  are  of  this  type. 


The  following  section  contains  three  lemmas  which  are  the  main 
results  of  this  paoer.  The  first  lemma  gives  an  upper  bound  on  the 
loss  in  objective  function  change  as  the  result  of  adding  a  cut  to  LP(1) 
which  is  not  the  strongest.  This  upper  bound  is  known  without  additional 
computation.  The  second  lemma  gives  a  lower  bounding  test  for  deleting 
columns  from  (1)  at  each  minor  iteration.  The  third  lemma  interprets 
the  test  of  lemma  2  and  shows  that,  in  the  limit  as  the  generalized 
prograrmiing  algorithm  converges  towards  an  optimal  integer  solution,  the 
lower  bound  test  will  delete  all  columns. 

The  final  section  contains  a  flow  chart  of  the  generalized  programming 
algorithm  and  a  summary. 

Reformation  and  Manipulation  of  IP  Problem 

Once  an  LP  basis  B  has  been  selected,  the  generalized  programming 
algorithm  continues  with  a  reformulation  of  IP(1)  which  we  now  review. 
The  reformulation  involves  additional  conditions  implied  by  the  fractions 
of  B'  .  In  particular,  the  columns  of  B"  with  addition  modulo  1 
generate  a  finite  abelian  group  of  order   D  =  |det  B|,   A  matrix 

diagonalization  method  is  used  to  find  integers  q.  >_2,  q.|q.^, ,  i=l,..., 

r 
r,    n  q.  =  D  such  that  G  can  be  represented  as  D  r-tuples  x.  = 

T=l  ^  ^ 

(x,,  ,. . .  ,A  .  ),  where  X..  is  integer,  0  lX^..<_q^-l.  For  convenience, 

let  x^  =   0.  Each  non-basic  a-  is  thus  mapped  into  such  an  r-tuple 

a.  =   (a, .,. . . ,a  . ).  and  the  requirement  that  the  basic  variables  be 

integer  is   z    a.x.   =  &   mod  (q,,...,q  ).  For  our  purposes  here, 

j=m+l  ^  ^  I     r 

it  is  convenient  to  use  the  following  equivalent  representation  of  a.. 

Each  unit  vector  e.,  i=l,...,m,  is  mapped  into  an  element  e-  e  6,  and  it 

^  m 

can  easily  be  shown  that  a.   =       z     a..e.  mod  (q,,...,q  ),  j=m+l,...,n. 


With  this  background,  we  can  rewrite   IP(1)   in  the  form:     Find 


min  z^  +     I         '-j^'j 
°       j=m+l 


(2a) 


n       _  — 

s.t.       z       a.  .X.   <_  b   ,   i  =  l ,...  ,m  (2b) 

j=m+l     ^^  ^         ^ 

n 

I        a.x.   =  6  mod   (q,  ,. . .  ,q   )  (2c) 

j=m+l       ^  ^ 

X.  non-negative  integer,  j=m+1,...,n  (2d) 

A  solution  X.,  j  =  m+1,...,n,  to   (2)  yields  the  solution  x.    ,  j=l,...,n, 

J  _        _  n  _   _  ^ 

to   (1)  by  setting  "i^.   =  b.   -       i  a.-x-    ,  i=l,...,m. 

^         ^       j=m+1         ^^  ^ 

The  first  step  in  the  IP  analysis  is  to  partition  the  set  {3.}1_     -, 
into  the  subsets 


'k  =  ^^j^    -l^  'ifi  ^\"^°^  (qr-.-.q,)>  0). 

k=0,l,...,D-l. 

Special  attention  must  be  given  to  the  set  (B  corresponding  to  the  right 
hand  side  element  3  in  (2c).  It  is  easy  to  show  that  a  e  (B  such  that 
*b.  -  a.  >_  0,  i=i,...,m,  can  be  deleted.  This  is  because  the  least  cost 
feasible  integer  solution  involving  a   at  a  positive  level  is  the  solution 
x^  =  1 ,  x^.  =  b.  -  a.  ,  i=l,...,m;  all  other  variables  set  to  zero.  This 
solution  costs  Zq  +  c  and  any  feasible  integer  solution  with  a  at  a 
positive  level  must  cost  at  least  z«  +  c  because  c.  >_0,  j=m+l,...,n. 
Thus,  we  delete  all  such  a  f rom  {B  making  sure  to  reel  ace  x  and  z  by  the 
least  cost  of  these  solutions  if  the  least  cost  is  less  than  z. 


Since  we  cannot  solve  (2)  directly,  we  replace  it  with  an  easier 
problem.  The  underlying  philosophy  is  to  relace  (2)  by  an  LP  relaxation 
which  can  be  easily  solved  and  provides  lower  bound  information  for 
adding  cuts  and  deleting  activities.  The  strongest  LP  relaxation  is 
the  following.  Let 

1=  ^^\+-i^'--'\)\(\+-['--'>\)   satisfies  (2c)  and  2d} 
We  use  generalized  programming  to  solve  the  following  LP  problem: 

Find 

•-  -  "^       _ 

z,   =  min  Zq  +       E       c-x.  (4a) 

j=m+l     ^  ^ 
n      _  — 

s.t.         z       a.  .X.   <.b     i=l,...,m  (4b) 

j=m+l     ^-^  "^         ^ 

_c 
(x^+T....,xJ   e:  X  (4c) 

where  X^  denotes  the  convex  hull  of  J.     Since  (4)  is  a  relaxation 

of  (2),  it  should  be  clear  that  z-,  <_  z*. 

Other  relaxation  manipulations  of  (2)  fall  into  three  categories. 

The  first  is  to  omit  some  of  the  (2b)  constraints  in  (4b).  For  example, 

a  simple  rule  would  be  to  include  in  (4b)  only  those  constraints  which 

are  violated  when  all  of  the  constraints  in  (2b)  are  ignored.  The  problem 

consisting  of  (2a),  (2c),  (2d)  is  the  unconstrained  grouo  optimization 

problem  of  [10]. 

The  second  category  of  relaxation  manipulations  is  to  replace  (2c) 

n     r  r 

by  some  new  system  of  congruences,    E    {14'.  (a..)}x.=  z   4'.  (6.  )niod(q-, , 

j=ni+i  i=i  1   iJ   J  i=i  1  T      ' 

...,q  )  where  each  4*.  is  an  endomorphism  on  Z  ,  the  cyclic  group  of  order 

q. .  Many  generalizations  of  this  idea  are  studied  by  Gomory  and  Johnson 
in  reference  9. 


The  third  category  of  relaxation  manipulation  results  from  an 
alternative  reformulation  of  IP(1 )  qiven  in  section  4  of  reference  11. 
This  reformulation  involves  a  change  in  the  data  and  the  number  of 
constraints  of  (2b),  and  a  change  in  the  system  of  congruences  (2c). 
An  LP  relaxation  similar  to  (4)  can  then  be  constructed.  As  with  the 
second  category,  this  manipulation  is  designed  to  deal  with  LP  bases  B 
with  large  determinants  D,  say  D  >  10,000.  The  reader  is  referred  to 
11  for  more  details. 

We  return  now  to  the  development  of  the  generalized  programming 
algorithm  based  on  the  relaxation  (4)  to  the  IP  problem  (2).  We 
emphasize  that  at  the  start  of  any  major  iteration  the  algorithm  has 
the  option  to  use  any  of  the  above  mentioned  relaxations. 

Minor  iteration  S  begins  with  the  solution  of  the  master  LP:  Find 


^    "    -   s 
Zq  +  min  z    e   (c.x/)  o^  (5a) 


,m  (5b) 


s=l  j=m+l   J  J    ^ 
S   n    _ 

s.t.   z   z    (ai-i^-i^)  ^c  <  b  i=l, 

s=l  j=m+l   ^J  J    ^    ^ 
S 

^     0  =1  (5c) 

s=l  ^ 


a^   >_  0,s=l,...,S  (5d) 

1    s  — s 

The  solutions  x  ,...,x  ,  have  been  previously  generated.  Let  a  ,  s=l,...,S 

be  an  optimal  solution  to  (5),  and  let  u. ,i=l,.,.,m  be  the  optimal  shadow  prices 

_S 
on  rows  1 m  in  (5b),  and  let  0  be  the  optimal  shadow  price  on  the 

convexity  row  (5c).  The  next  step  is  to  reorder  the  elements  a.  e  A^  by 

ascending  value  of  c.  +  u  ci . ,  and  define 


mi 


n  {C.+u  a.:     j  such  that^j^^k)       if  A  j'*   (6) 


t,^(u^)  =  L      -  ifA|,=*. 

k=0,l,...,D-l. 

If  t.  (u  )  <  0  for  any  k,  select  any  a.  e  Aj^  with  c.  +  u  a •  =  t|^(u  )  and 
add  a  column   (s-i  ^  ,a2 ,•,•••  .a^ J     with  cost  coefficient  c-  to  problem  (5) 
with  variable  m-  constrained  to  be  non-negative.     Then  problem  (5)  is 
t^eoptimized.     Strictly  soeaking,  all   columns  added  to  the  master  LP(5) 
must  be  retained  in  order  to  protect  against  cycling.     As  a  practical 
matter,  the  only  columns  which  need  be  retained  in   (5)  at  each  iteration 
are  the  optimal   basis  columns. 

_c 
Suppose  t.  (u   )  >^  0,k=0,l ,. . .  ,D-1 .     Then  the  following  group  optimiza- 

_c 

tion  problems  are  solved:  For  k=0,l  ,2,. . .  ,D-1 ,  find  GC^i.;  u  )  where 


^       D-1     c; 
G(x.  ;  u^)  =  min  z     t  (u^)y  (7a) 

D-1 
s.t.   z  Xj,y^  =  X|^  mod  (q^,...,q^)  (7b) 

1=0 

Z  y>l.  (7c) 

1=0      ^ 

y     non-negative  integer,  i=  0,1,..., D-1  (7d) 

Constraint  (7c)  needs  some  explanation.     For  ^y^^O,   (i.e.  for  k=l  ,2,. . .  ,D-1 ), 
This  constraint  is  non-binding.     For  the  case  Xy=  0,  the  constraint  (7c) 
imposes  the  condition  that  the  otherwise  feasible  solution  in   (7)  y^=  0, 

c 

«,  =  0,  1,...,D-1,  is  not  allowed.  This  improves  the  lower  bound  6(0;u  ) 
of  lemma  2  given  in  the  next  section. 


em 


Before  discussing  further  minor  iteration  S,  we  note  that  probl 
(7)  provides  us  with  a  valid  Gomory  cut  for  any  u  such  that  t  (u)  >_0, 
z  =  0,1,..., D-1.     This  cut  is 


where 


and 


n 

E         p.(u)    X.    >_  Pf,(u) 

j=m+l     J  J        ^ 


P,-(u)  =  t  (u)  for  A     containing  a., 

J  X.  X.  J 


(8) 


Pq(u)  =  G(6-,u). 
.  The  cut  (8)  for  u  optimal  on  rows  (4b)  is  slightly  different 
from  the  one  given  in  [16].  The  difference  is  that  here  t  (u)  has  replaced 

c.  +  u  a.  for  a.eA  when  t  (u)  <  c.  +  u  a..  In  so  doing,  the  new  cut  (8) 

J     J     J  *■      *•      J     J 

is  stronger  than  the  one  in  [16], 


We  also  mention  that  there  is  no  guarantee  that  Pq(u)  >  0,  and  thus  no 
guarantee  that  (8)  cuts  off  the  previously  optimal  LP  solution.  This  weak- 
ness can  be  remedied,  but  again  as  a  practical  matter,  the  generalized 
programming  algorithm  should  probably  be  abandoned  (perhaps  temporarily) 
in  favor  of  branch  and  bound  when  the  method  above  yields  nothing  but  non- 
cutting  cuts  as  u  coverages  to  u  optimal  for  (4b). 

Mathematical  convergence  in  a  finite  number  of  iterations  of  the 
generalized  programming  algorithm  could  be  assured  by  adding  at  each  major 
iteration  a  Gomory  cut  such  as  the  one  given  on  page  524  of  reference  2 


10 


whenever  the  algorithm  tries  to  add  a  cut  (8)  with  Pgdj)  =  0.  Computational 
experience  with  the  original  version  of  the  cutting  plane  algorithm 
clearly  indicates,  however,  that  such  mathematical  properties  of  an 
algorithm  may  have  little  to  do  with  computational  efficiency. 


_c     _c 

If  G(6;u  )  <  0  ,  then  in  principle. a  new  column  is  added  to  the  master 

S+1 
LP(5)  and  it  is  reoptimized.  The  solution  x   used  to  generate  the  column 

is  derived  in  the  obvious  fashion  from  the  optimal  solution  to  problem  (7) 

S+1 

with  right  hand  side  6  in  {7b).  It  is  easy  to  show  that  if  x.   is 

feasible,  then  the  new  column  will  come  into  the  basis  at  a  level  of  one. 

S+1 
Thus,  it  is  not  necessary  to  explicitly  test  x   for  feasibility  but  simply 

to  check  if  the  new  LP  solution  satisfies  this  property. 

If  G(e;u  )  =  0  and  x  =  z  a  x^  is  integer,  then  x^  is  optimal 

s=l  ^ 
in  (2).  Otherwise,  the  procedure  can  only  continue  by  adding  a  cut 

to  the  optimal  tableau  of  LP(1),  reoptimizing  it  and  repeating  the  above. 

Of  course,  it  may  be  desirable  because  of  slow  convergence,  to  add  a  cut 

before  the  termination  condition  G(3;u  )  =  0  is  achieved.  The  loss 

in  objective  function  increase  resulting  from  the  use  of  a  non-optimal 

cut  can  be  measured  by  dual  lov/er  bounds  which  we  discuss  in  the  next 

section. 


Deletion  of  Activities  by  Lower  Bounds 

The  dual  relation  of  LP(4)  to  IP(2)  (equivalently,  IP(1)),  and 
also  that  of  the  generalized  programming  algorithm  for  solving  LP(4) 
to  LP{4),  can  be  used  to  construct  lower  bounds  which  allow  activities 
to  be  deleted  at  each  minor  iteration.  The  lower  bounds  are  also 
useful  in  deciding  when  to  terminate  a  major  iteration  with  the 
addition  of  a  new  cut,  followed  by  an  LP  reoptimization. 


n 

The  first  result  provides  upper  and  lower  bounds  on  the  increase 
of  LP  objective  function  as  the  result  of  the  addition  of  a  cut   (8)  for 
optimal   dual   variables  u     from  any  master  LP(5).     The  augmented  LP  is:     Find 


-    -S  -  "      - 

z,  (u  )  =  min  z^,  +      z      c-x.  (9a) 


n 
s.t.         z     a.  .X.  <_  b.,i=l,...,m  (9b) 

j=m+l     ^^  ^         ^ 

z      p.(U^)x.  iPn(u^)  (9c) 

j=m+l     ^  -^        ^ 

X.  >.0,  j=m+l,...,n  (9d) 

Lemma  1.  Suppose  for  an  arbitrary  feasible  set  of  columns  in  the 

—   -S  — 
master  LP(5)  that  c-  +  u  a.  >^  0,  j=m+l,...,n,  for  the  vector  of  optimal 

simplex  multipliers  u  .  Then,  the  following  inequalities  hold 

Wg  >  z"-|  1  Zi(iJ^)  1  w^  +  G(6;u^)  -  0^  (10) 

Proof:  The  left  most  inequality  is  obvious  from  the  relation  of  the  master 

LP(5)  to  LP(4).  The  inequality  z,  i  z-i  (u  )  follows  from  the  fact  that 

any  (x  -,,..., x  )  feasible  in  LP(4)  with  minimal  cost  z,  must  satisfy  the 

constraints  of  problem  (9).  To  see  this,  it  is  important  to  note  only  that 

the  inequality  (9c)  must  hold  for  all  XeT  and  therefore  for  all  xeX^. 

To  establish  z^ (u^)  ^  w^+G(B;u^)  -  e^,  consider  the  dual  to  LP(9): 

_    m   _         5- 
max  £q  -  _z^u.b.  +  u^^^Pq(U  ) 

m  5-    _ 

s.t.  -  _z^  u.a-  .  .  u^,tP.(u  )^Z.   j=m+l....,n         (11) 

u.  >^  0,  i=l ,. . .  ,m+l . 

The  desired  inequality  can  be  established  by  noting  that  the  solution  u.  =  u.  , 

— S  ■■•       ""S  — 

i=l,...,m+l,u     ,=1,  is  feasible  in   (11)  because  p.(u  )  ic  +  u  a  •,j=m+l ,. . .  ,n 
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(See  (6)  and  (8).)  By  LP  duality,  therefore, 

z-|  (u  )  1  Zq  -  u  b  +  Da(u  ).  But  Zq  -  u  b  =  w  -  0  because  u  , 

-S  S  -S 

0    are  optimal   dual   variables  for  the  master  LP(5),  and  Pq(u  )  =  G(B;u  ). 

This  establishes  the  desired  inequality  and  completes  the  oroof. 

The  next  result  provides  us  with  a  simple  test  for  deleting  activities 
at  each  minor  iteration. 

Lemma  2.  Let  z  be  an  upper  bound  on  the  minimal  cost  z*  of  IP(1). 

Let  u  be  the  optimal  simplex  multipliers  for  the  master  LP(5)  with  an 

-S- 
arbitrary  set  of  columns,  and  suppose  c.  +  u  a-  >_0,  j=m+l,...,n.  Then, 

without  loss  of  equality,  all  a  e  A  such  that 

c^   +  u^ij^  i  z  -  {Zq  +  G(g-X^;  u^)  -  u^B) 

can  be  deleted  from  problem  (1). 

Proof:     Consider  the  following  subproblem  derived  from  (2).     Find  z*  where 
z*  =  min  io  .  E^  .   _z^^^   -c...  (13a) 


s.t.       z        a..x.  <_b.   -  a..,  i=l,...,m  (13b) 

j=m+l     '^  ^         '         ^^ 


n 

I 
j=m+l 


I         a.x.   =  &  -  \^  mod   (qi,...,q^)  ('13c) 


X.  non-negative  integer,  j=m+l,...,n  (13d) 


The  cost  z*  in  (13)  is  the  minimal   cost  of  using  activity  a^  at  least  once. 
We  prove  the  lemma    by  showing  that 
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W-,   A.  -   j_  r-/n  I   .  r.J\    ~->L 


z*  1  c^  +  u  a^^  +  ig  +  G(e-A^;  u^)  -  G^b  (14) 


The  right  hand  side  of  this  inequality  is  by  assumption  at  least  as  great 
as  the  upper  bound  z  on  the  optimal  cost  of  (1),  and  thus  if  (14)  holds  we 
can  conclude  that  the  activity  a  can  be  (jeieted. 

'    The  minimal  cost  of  problem  (13)  with  che  constraints  (13b)  ignored 
.Q  +  (T  +  G(b-Xj^;  u^).  The  case  A^= 


-    ^         -S 
is  Zn  +  c  +  G(b-x  ;  u  ).  The  case  A„=<B  is  special  because  of  the  constraint 


(7c)  which  affects  the  lower  bound  G(0;  u  ).  Recall  thatiCB  was  constructed 

so  that  for  any  a^^e  (B,  i.e.  B-x  =  0,  at  least  one  of  the  m  constraints  in  (4b) 

is  violated;  that  is,  b.  -  a"-,<  0  for  at  least  one  i.  It  is  correct, 

therefore,  to  require    i     x.  >  1  in  oroblem  (13),  and  the  lower  bound 
j=m+l  ^ 

G(0;  u  )  is  correct. 


Let  X*  be  an  optimal   solution  to  problem  (13)   (if 
(13)  is  infeasible,  then   (14)  holds  trivially).     The  solution  x*  satisfies 
(13c)  and   (13d),  and  therefore 


n  m 

+ 


c„   +     Z         (E.  +     Z     u^i.Jx^  ^E     +  G(6-X    ;   G^)  (15) 


j=m+l      ^       i=l 
Moreover, 


I        a     x^  1  B.   -  a.    ,  i=l m, 

j=m+l     ^^  ^         ^         ^^ 


and  since  u^  >_  0,  i=l ,. . .  ,m,  we  have 


"^     -S     "         -       .        "!     -S, 


z     D^     I         i     x^^    z     u^(b     -  a.    )   .  (16) 

i=l     T   j=ni+l     ^^  J       i=l     ^     ^         ^^ 
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Combining  (15)  and  (16).  we  have 


n         "^   s 


or,  adding  Zq  to  both  sides  and  rearranging  terms. 

But  the  left  hand  side  of  this  inequality  is  z*  which  is  what  we  wanted  to 
show.  This  completes  the  proof. 

The  lower  bound  on  the  right  hand  side  of  (12)  consists  of  the  term 

'  ^   -'  S   -  -  S 

z  -  Zq  which  is  non-neqative,  and  the  term  G(e-A  •  u  )  -  u  b  which  may  have 

-  S 
either  sign.  As  the  vector  u  converges  to  a  vector  of  optimal  dual  variables 

for  LP(4),  however,  there  is  some  regularity  of  behavior  of  this  lower  bound. 

Lemma  3.  Suppose  at  iteration  S  that  the  optimal  dual  variables 

-S  -S  -S    -S 

u  ,  0  found  for  the  master  LP(5)  satisfy  G(b;u  )  =  Q  and  therefore  that 

_c 

u  is  a  vector  of  ootimal  dual  variables  for  the  constraints  (4b)  and  also 

that  an  optimal  solution  to  LP (4)  has  been  found.  Then,  without  loss  of 

optimal ity,  activity  sl  zh     can  be  deleted  if 

E,  +  D^a",  -  t^(u^)  i-z  -1^,  (17) 

where  z-i  is  the  minimal  objective  function  cost  in  LP(4). 
Proof:  We  prove  the  lemma  by  demonstrating  that 

"z  -  z^  +  t^(u^)  >?  -  {Zq  +  G(B-X^;  Ii^)-lj^b}, 
and  therefore  the  activity  can  be  deleted  by  the  result  of  lemma  2.  Now, 
problem  (7)  is  a  shortest  route  problem  and  therefore 

G(B;u^)  Itju^)  +  G(S-A^;  J^).  (18) 


T5 


Thus,  we  have 


z  -  z^  +  t^(u^)  I'z  -  z"j+  G(6;u^)  -  G(b-x^  ;  u  ) 


(19) 


By  lemma  1,  it  is  clear  that  z^  =  w^  since  G(6;  u  )  =  0  ,  and  since  w^  = 

c    c-  ■       -  S 

Zf^  +  0  -  u  b,  the  right  hand  side  of  (19)  is  equal  to  z  -  Zq  -  0 

+  G(B;  u^)  -  G(6-A  •  u^)+u%.  The  lemma  is  proven  by  recognizing  that 

-  <;      — "^ 
-0  +  G(e;  u  )  =  0  in  this  last  expression. 

The  reader  may  have  noticed  that  when  (18)  is  an  equality,  the 
tests  (12)  and  (17)  are  identical.  When  (18)  is  not  an  equality,  then 
(12)  will  delete  a  greater  number  of  activities  because  the  right  side 
of  (12)  is  less  than  z  -  i^  +  t^(u^). 

The  result  of  lemma  3  also  gives  us  a  qualitive  insight  about  the 
difficulty  of  distinguishing  between  columns  which  are  very  similar.  As 
the  generalized  programming  algorithm  continues  through  major  iterations, 
the  term  z  -  z-i  converges  towards  zero.  It  may  be,  however,  that  two  activities 
a.  and  a-  are  so  similar  that  they  remain  in  the  same  equivalence  class 

J-i        Jo 

-    -S-     -     -S-       -S 
A  and  c.  +ua.  =c.  +ua.  =t(u)  through  many  major  iterations.  This 

*-     J     '^1    "^2     Jo    ^ 
can  be  the  case,  for  example,  if  c  =  c  and  a.  and  a.  differ  only 

J-]        J2  Jt  J2 

on  non-binding   (in  the  LP  sense)  rows   in   (1).     This  phenomenon  has  been 
observed  on  large  airline  crew  scheduling  problems  where  the  similarity  of 
many  subsets  of  rotations   (activities)  makes  LP(1)  and 

IP(1)  difficult  to  solve  optimally. 


Summary 

Figure  1  is  a  flow  chart  of  the  generalized  proqraiming  algorithm 
outlined  above.  Steps  ml  through  m6  constitute  a  minor  iteration,  and 
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steps  Ml  through  M5  constitute  a  major  iteration.  It  is  important  to 
emphasize  again  the  flexibility  in  problem  manipulation  inherent  in 
our  approach.  First  of  all,  there  is  flexibility  in  the  choice  of  group 
structure  obtained  in  step  M4.  Second,  there  is  flexibility  in  the 
choice  of  rows  to  include  in  the  master  LP(5)  of  step  m5.  Finally,  if 
G(6;u)  <  0  in  step  m4,  it  may  still  be  desirable  to  add  a  cut  to  the 
LP  (see  lemma  1 ). 

It  is  premature  at  this  time  to  discuss  in  any  detail  the  possible 
computational  efficiency  of  this  algorithm.  We  simoly  point  out  that 
all  of  the  sub-algorithms  required  in  the  construction  of  a  system  based 
on  figure  1  have  been  realized  and  extensively  tested  as  computer  codes. 
Representative  times  for  M4  and  m2  which  are  the  group  theoretic  algorithms 
are  given  in  reference  12.  Typical  times  on  the  IBM  360/85  for  IP 
problems  up  to  300  rows  are  in  the  range  of  .1  to  10  seconds,  and  these 
times  could  be  reduced  further  if  the  sub-algorithms  were  to  be  used 
repeatedly  on  a  given  problem.  There  is  also  fairly  strong  evidence  that 
group  optimization  problems  provide  strong  lower  bounds  ([10;  p.  301]). 
The  other  sub-algorithms  are  linear  programming  algorithms. 

An  experimental  code  is  currently  being  built  using  the  6LM  approach 
of  [16]  to  generate  strong  cuts.  Once  this  code  is  comoleted,  the 
extensions  proposed  here  will  be  possible.  It  is  important  to  mention 
that  the  design  of  a  system  based  on  figure  1  would  have  to  include  a 
variety  of  efficient  sorting  methods  for  handling  the  many  columns  stored 
in  secondary  storage  which  are  frequently  tested.  The  reader  is  referred  to 
Martin  [14]  for  a  complete  survey  of  these  methods. 
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Experimental  evidence  so  far  with  the  method  of  [16]  indicates 
that  the  addition  of  cuts  causes  the  absolute  value  of  the  determinants 
of  optimal  bases  to  increase.  Thus,  the  relaxation  method  of  section  2 
in  reference  11  for  reducing  these  values  after  an  optimal  LP  basis  has 
been  obtained  would  appear  to  be  necessary. 

Finally,  we  mention  that  the  generalized  programming  algorithm  can 
be  specialized  in  a  straightforward  way  to  handle  explicit  integer  upper 
bound  constraints  on  the  variables  (e.g.,  upper  bounds  of  1).  The 
only  charge  of  consequence  is  the  possibility  of  using  the  zero-one 
group  optimization  algorithm  given  in  reference  10.  to  obtain  tighter 
bounds.  We  point  out  that  lemma  3  no  longer  holds  if  the  zero-one 
group  problem  is  used  because  (18)  is  not  valid  for  that  problem. 


19 


[1]   Abadie,  J.  (Editor).  Integer  and  Nonlinear  Programming,  American 
Elsiver,  1970. 

[2]   Dantzig,  G.B.,  Linear  Programming  and  Extensions,  Princeton  University 
Press,  Princeton,  N.  J.  1963. 

[3]   Dzielinski,  B.P.  and  Gomory,  R.E.,  "Optimal  Programming  of  Lot  Sizes, 
Inventory  and  Labor  Allocations,"  Management  Science,  11, 
DP.  874-890  (July,  1965). 

[4]   Ford,  L.R.  and  Fulkerson,  D.R.,  "A  Suggested  Computation  for  Maximal 
Multi -Commodity  Network  Flows,"  Management  Science,  5, 
pp.  97-101  (October,  1958). 

[5]   Geoffrion,  A.M.,  "An  Improved  Implicit  Enumeration  Approach  for 

Integer  Programming,  Operations  Research,  17,  pp.  437-454. 
(May- June,  1969). 

[6]   Geoffrion,  A.M.,  "Elements  of  Large-Scale  Mathematical  Programming, 
Parts  I  and  II,"  Management  Science,  16,  pp.  652-691, 
(July,  1970). 

[7]   Gilmore,  P.C.  and  Gomory,  R.E.,  "A  Linear  Programming  Aporoach  to 

the  Cutting-Stock  Problem— Part  II,"  Operations  Research,  11, 
pp.  863-888,  (November-December,  1963)! 

[8]   Gomory,  R.E.,  "Some  Polyhedra  Related  to  Combinatorial  Problems," 

Linear  Algebra  and  its  Applications,  2,  pp.  451-558,  (1969). 

[9]   Gomory,  R.E.  and  Johnson,  E.L.^  "Some  Continuous  Functions  Related 

to  Corner  Polyhedra,"  RC  3311,  IBM  Research,  February  23,  1971, 
Yorktown  Heights,  N.  Y. 

[10]  Gorry,  G.A.,  and  Shapiro,  J.F.,  "An  Adaptive  Group  Theoretic  Algorithm 
for  Integer  Programming  Problems",  Management  Science,  17, 
pp.  285-306  (January,  1971). 

["11 J  ,  and  ,  and  Wolsey,  L.A.,  "Relaxation  Methods 

for"  Pure  and  Mixed  Integer  Programming  Problems,"  Sloan  School 
of  Management,  Working  Paper  456-70,  April,  1970.  (To  apoear 
i n  Management  Science. ) 

[12]  ,  and  ,  Northup,  W.D.  and  Zakovi ,  C.H.,  "A 

Group  Theoretic  Integer  Programming  Algorithm:  Systems 
Design  and  Computational  Exoerience,"  Sloan  School  of  Manage- 
ment Working  Paper  513-71,  February,  1971. 

[13]  Held,  M.  and  Karp,  R.M.,  "The  Traveling-Salesman  Problem  and  Minimum 

Spanning  Trees,  Operations  Research,  18,  pp.  1138-1162,  (1970). 


20 


[14]  Martin,  W.A. ,  "Sorting"  (to  appear  in  Computing  Surveys. ) 

[15]  Roy,  B.,  Benayoun,  and  Tergny,  J.  "From  S.E.P.  Procedure  to  the 
Mixed  Ophelie  Program,"  Chapter  20  in  [1]. 

[16]     Shapiro,  J.F.,   "Generalized  Lagrange  Multipliers   in   Integer 
Programming,"  Operations  Research,  19,  pp.   68-76, 
(January-February,  1971). 

[17]     Tomlin,  J. A.,   "An   Improved  Branch-and-Bound  Method  for  Integer 
Programming,"  Operations  Research,  19,  po.   1070-74, 
(July-August,  1971 ). 


3  TDflD  D03  b7D  3D1 


3  TDfiD  DD3  b7D  ET3 


^T5-7/ 


:ffo'7/ 


,f,T 


3    TOfiQ    003    b7D    251 


t--'\ 


HD28 

Nos.     3-'' 

v-^s.,    0- 


3    TOflO    DD3    701    3DM 


3    TOfiO    003    b70    2MM 


s^'-^l 


3    TOflO    003    701    31E 


'^(^'7/ 


